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Abstract 

Machines whose main purpose is to permute and sort data are 
studied. The sets of permutations that can arise are analysed by means 
of finite automata and avoided pattern techniques. Conditions are 
given for these sets being enumerated by rational generating functions. 
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1 Introduction 

From the earliest days of Computer Science abstract machines have been 
used to model computations and categorise them according to the different 
resources they require. In this paper we consider a new type of machine 
that is suited to modelling computations whose sole or main effect is to 
permute data. Unlike most classical machines these new machines have an 



infinite input alphabet whose symbols form the data that is to be permuted. 
Despite this we shall show how the theory of finite automata can be deployed 
in their analysis. 

A permuting machine is a non-deterministic machine with the following prop- 
erties: 

1. it transforms an input stream of distinct tokens into an output stream 
that is a permutation of the input stream, 

2. it is oblivious to the values of the input stream tokens, 

3. it has a hereditary property: if an input stream a can be transformed 
into an output stream r and a' is a subsequence of a whose symbols 
transform into the subsequence r' of r, then it is possible for a' (if 
presented as an input to the machine in its own right) to be transformed 
into an output stream r'. 

Examples 

1. A riffle shuffler divides the input stream into two segments and then 
interleaves them in any way to form the output stream. 

2. A stack receives members of the input stream and outputs them under 
a last- in- first-out discipline. 

3. A transportation network [1] is any finite directed graph with a node to 
represent the input stream and a node to represent the output stream. 
The other nodes can each hold one of the input objects and the objects 
are moved around the graph until they emerge at the output node. 

The oblivious property of permuting machines allows us to name the input 
tokens 1,2, ... ,n (in that order) in which case the output will be some per- 
mutation of 1, 2, . . . , n. In this way we can consider a permuting machine 
to be a generator of permutations (usually, because of the non- determinism, 
generating many of each length). There is another point of view which is 
sometimes more useful where we consider the input stream to be some per- 
mutation of 1,2, ... ,n and ask whether the machine is capable of sorting the 
tokens (so that they appear in the output stream in the order 1,2, ... ,n). 
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These two viewpoints are equivalent since a machine can generate a partic- 
ular permutation a if and only if it can sort the permutation . 

However, it is the hereditary property which allows non-trivial properties of 
permuting machines to be found because of a connection with the combi- 
natorial theory of involvement and closed sets of permutations. Formally, a 
permutation tt is said to be involved in another permutation a (denoted as 
TT ^ cr) if TT is order isomorphic to a subsequence of a. For example 231 is 
involved in 31542 because of the subsequence 352 (or the subsequence 342). 
We also say that a avoids vr if tt is not involved in a. 

Permutation involvement has been an active area of combinatorics for over 
10 years although it surfaced long before that in data structuring questions 
on stacks, queues and deques (see [7, 9, 10]). Involvement is a partial order 
on the set of all permutations and is conveniently studied by means of order 
ideals called closed sets. A closed set X of permutations is one with the 
property that a & X and n ^ a imply vr G A*. The connection between 
permuting machines and closed sets is via the following result which follows 
from the definitions. 

Proposition 1 The set of permutations that a permuting machine can gen- 
erate, and the set that it can sort, are both closed. 

In classical automata theory machines are associated with the languages they 
recognise. The above proposition suggests that the appropriate associated 
language of a permuting machine is the closed set of permutations that it 
can generate. We will study various permuting machines and their associated 
closed sets, and will show the utility of the permuting machine paradigm as 
a tool for advancing the theory of permutation involvement. Before giving 
further details of our results we recall some key concepts about permutation 
involvement. 

A closed set X is, by definition, closed "downwards" . But that is equivalent 

to its complement X'~'' being closed "upwards" {a G X'-^ and a ^ t imply 
r G X'-'). Obviously, X'-^ is determined by its set of minimal permutations 
which we denote by B{X) and call the basis of X. Clearly 

X = {a\a^ X^} = {(7 I TT 2< (7 for aU TT G B{X)} 

is determined by its basis. By definition, B[X) is an antichain in the in- 
volvement order and conversely every antichain has the form B[X) for some 
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closed set X. The bases of the closed sets of permutations generated by the 
machines in examples 1 and 2 above are {321, 2413, 2143} and {312} respec- 
tively. The closed sets that arise in practice are generally infinite so it is 
clearly significant to know when a finite description is available by means 
of the basis. Indeed, many combinatorial enumeration investigations begin 
from some particular finite basis and study properties of the closed set that 
it defines ([4, 11]). We let A{B) denote the closed set whose basis is the 
antichain B; in other words 

A{B) = {(7 I /? 2< (7 for all /? e B} 

Given a closed set X (or a permuting machine that defines it) we would like 
to be able to solve 

• The decision problem: given a permutation a decide whether a & X 
(in linear time if possible), 

• The enumeration problem: determine, for each length n, the number 
of permutations in X , 

• The basis problem: find the basis of A", or at least determine whether 
the basis is finite or infinite. 

In this paper we shall show how to exploit the classical theory of finite au- 
tomata to make progress on these problems. To do this we have to over- 
come the difficulty that this theory deals with strings over a finite alphabet, 
whereas the strings of X are written in the infinite alphabet 1,2,.... There- 
fore we shall look for encodings of the permutations in X as strings over 
a finite alphabet (normally [k\ — {1, 2, . . . , /c}) and hope to prove that the 
language of such encodings is regular (or to find conditions under which this 
is so). Once we have proved the regularity of such a language we can appeal 
to two well-known facts: that regular languages have linear time recognisers, 
and that the generating function (the formal power series whose coefficients 
give the number of sequences of each length) is a rational function. 

Of course this approach cannot be expected to succeed in all cases if for no 
other reason than that closed sets do not always have rational generating 
functions. Nevertheless, in Sections 2 and 3, we shall give two wide classes of 
closed sets (and permuting machines) which show that the approach can have 
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significant successes. In particular we produce infinite famifies of closed sets 
all of whose finitely based closed subsets have rational generating functions. 
Our results therefore link to the many recent papers where particular closed 
sets have been enumerated (for example, [3, 5, 8]). In the final section we 
indicate how we hope our approach may be extended. 

We conclude this section by recalling some basic facts about transducers. 

For our purposes a transducer is essentially a (non-deterministic) finite au- 
tomaton with output symbols (from an alphabet F) as well as input symbols 
(from an alphabet A). We allow e inputs as well as e outputs. A transducer 
defines a relation between A* and F* in a natural way. That is to say, for 
every path in the transducer from the starting state to one of the final states, 
let the sequence of input labels be a and the sequence of output labels fi (all 
e's being omitted of course); then (a,/?) is a related pair. 

In any transducer we can interchange the input and output symbols on each 
transition to obtain another transducer. Therefore 

Lemma 1 IfTZisa transducer relation so also is the transpose relation TZ*. 

Let £ C A* and define 

Cn^{(3er* \ there exists a e C with (a, /3) e 7^} 

The main result we need from the theory of transducers appears as exercise 
11.9 in [6]. For completeness, and to establish notation, we include the proof. 

Proposition 2 IfTZisa transducer relation and C is a regular subset of A* 
then jCTZ is regular. 

Proof: Let P be the set of states of the transducer, /i the transition function 
(mapping P x (A U {e}) into subsets of P x (F U {e})), po the initial state, 
and E the set of final states. 

Let M be a finite automaton recognising C. Suppose that M has set of states 
Q, transition function 5, initial state go, and set of final states F. Extend 
the definition of S so that (g, e) i— > g is a valid transition for all g e Q. 

Now define an automaton as follows. The set of states is P x Q, the initial 
state is (po,?o), and the set of final states is E x F. The transitions are 
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defined as follows. If there are transitions 

d,g 

Pi ^P2 

and 

d 

qi q2 

(where pi,P2 G P, qi,q2 & Q, d e AU {e} and g e T L) {e}) then A'' has a 
transition 

{pi,qi) ^ {P2A2) 

We prove that the new automaton recognises the set CTZ. Let (3 be any string 
in CR. By definition of CTZ we may choose a string a e £ with (a, /3) G TZ. 
Then we have transducer transitions 

01, bl 02,62 On,bn 

Pq ^ Pi ^ ^ Pn 

with Pn & E witnessing that (a,/?) e i?. Then we have a — ai...a„, 
(3 = hi . . .hn (where, possibly, e symbols may occur) . We also have transitions 
of M 

ai 02 a„ 

<?o ^ g2 — ^ > gn 

with Qn & F witnessing that a & C Then, by definition, we have transitions 

(Pi_i,gi_i) iPi,qi) 
in demonstrating that P is accepted by A^. 

We reverse this argument to get the converse. Suppose that /3 e F* is 
accepted by N via a sequence of transitions 

(pj_i,gi_i) iPi,qi) 

where (3 — hi . . .hn with each 6^ e F U {e}. By definition of N there exist 
ai, . . . , On e A U {e} and state transitions 



of the transducer, and transitions 

qi-i qi 

of M. This proves that a = ai . . . a„ e £ and (a, (3) as required. 
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2 Bounded classes 



In this section we consider permuting machines as 'black boxes' into which 
input tokens are inserted and from which they eventually emerge as output 
tokens. So, at any point of a computation there may be some tokens which 
are 'inside' the machine (in the machine's memory) awaiting output. The 
chief hypothesis of this section is that, for some constant k, the machine can 
contain no more than k tokens at a time (so if it is full to capacity it must 
output a token before further input is possible). Such machines are said to 
be fc-bounded. 

If we consider a A;-bounded machine as a generator of permutations then no 
permutation of length k + 1 that begins with k + 1 can be generated from the 
input 1,2,..., k+1. Thus the closed sets associated with A;-bounded machines 
are subsets of the closed set whose basis consists of the k\ permutations 
k + l,ai, . . . ,ak where ai, . . . , ranges over all permutations of 1,2, . . . , k. 

We shall see shortly that permutations in fl^ may be encoded as words in a 
/c-letter alphabet. Anticipating this, we define a subset of flk to be regular 
if its encoded form is a regular set. We shall show that a closed subset X of 
Qfc is regular if and only if its basis is regular. The proof of this result is, in 
principle, constructive in the sense that a recognising finite automaton for X 
can be built from one that recognises its basis and vice versa. In the course 
of proving this result we shall prove that it is decidable whether a regular 
subset of Qfc is a closed subset. 

Let TT = ni7i2 . . . 7r„ be a permutation of length n. Its rank encoding is the 
sequence 

where 

Pi = \{j \j>i, TTj < TTj}! 

is the rank of tt^ among {tTj, tTj+i, . . . , 7r„}. 

Obviously, tt e ilfe if and only if 7ri7r2 . . . 7r„ has no subsequence of length k + 1 
whose first element is the largest in the subsequence and this is precisely the 
condition that Pi < k for all i. Thus every subset of encodes as a subset 
of [k]*. 

Proposition 3 flk is regular. 
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Proof: It is easy to see that a word p — piP2 . . . p„ is the encoding of some 
permutation if and only if 

Pn+i-i < i for all i (1) 

(and, if this condition holds, the permutation can readily be calculated). 
In fact, for p e [k]* the above inequalities may fail to hold only for i — 
l,2,...,/c — 1. Let F be the set of all words of length at most A; — 1 for which 
(1) does not hold. We now have 

EiQk) = [k]* \ [k]*F, 
which is a regular set. ■ 



Example 1 Consider the closed subset X of whose basis is 312, 321, 231. 
The first two basis elements ensure that, indeed, X C Q2 the permutations 
of X encode as words in the alphabet {1,2} and end with a 1. It is readily 
checked that the third basis element restricts these words by prohibiting 
consecutive occurrences of the symbol 2. 

The set of words that do contain consecutive 2s is described by the regular 
expression [2]*22[2]* and so is regular. However E(X) is the complement of 
this regular set within the regular set E{Q2) and so is also regular. Thus X 
is a regular closed set. The generating function of E{X) is well-known to be 

1 

1 — X — x'^ 

and, since E is one-to-one, this is also the generating function of X. 

This easy example serves to illustrate that the condition of avoiding a permu- 
tation translates into restrictions on encodings although they are generally 
much more complicated than the ones above. The argument that proves 
regularity is a very special case of more general arguments to come. 

Transportation networks arc another source of examples. Theorem 1 of [1] 
proves that the closed sets associated with these are all regular. That paper 
also contains an example to show that regular closed sets need not be finitely 
based. 

We also note that not every closed subset of is regular. Indeed, as shown 
in [2], there are uncountably many closed subsets in fl^, if ^ > 3; but there 
are only countably many regular languages over [k]. 
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2.1 A transducer to delete a letter from a word 

Let TT = 7ri7r2 . . . 7r„ be a permutation in flk and let p = piP2 ■ ■ - Pn be its 
rank encoded form. Let tt' be the permutation obtained from tt by deleting 
TTj (and relabelling appropriately) and let p' — . . . . be its 

encoded form. We put 

diP = p' 

call this the ith derivative of p. The process of passing from p to p' is 
called deleting a letter from p. We shall show how this may be done without 
"looking at" tt. 

Example 2 Let p — 2331211 representing the permutation tt = 2451637. 
Then removing the 6th clement of tt results in the permutation tt' = 234156 
whose encoding is p' — 222111. 

Suppose we have to delete the ith letter from p. We compute p' by scanning p 
from the right. For the positions to the right of pi, each pj represents the rank 
of some element of tt among its successors, so these ranks will be unchanged 
by the deletion. Therefore until we reach pi itself (which we delete) nothing 
changes. But for j < i we need to know whether or not ttj > TTj (so that we 
can tell whether or not to reduce pj by 1). To do this we keep track of a 
variable rj defined as the rank of tTj in the set {tTj+i, . . . , 7r„} (the number of 
symbols in this set that are less than or equal to tTj). Clearly 

TTj > Tii if and only if pj > rj 

Provided we have rj we can decide whether wc should reduce pj. But, as the 
pointer j moves to the left, we can easily update rj. Clearly, if ttj > ni then 
Tj-i = Tj] and if nj < TTj then rj-i = rj + 1. We therefore get Algorithm 1. 

Two easy observations make this into a finite state algorithm. The first 
is the natural programming trick to use a single variable r in place of rj. 

The second looks odd as a programming trick but is nevertheless essential. 
When Vj > k the first alternative of the if is not followed nor is it followed 
thereafter; so we 'freeze' rj to the value k once it reaches k. The result is 
Algorithm 2. 

It is now easy to define a transducer for the relation 

V — {{p,p') I p' is obtained by deleting one letter from p} 
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Algorithm 1 First form of tlie deletion algorithm 
for j := n downto i + 1 do 

P'j ■= Pj 
end for 

n-i := Pi 

for j :— i — 1 downto 1 do 
if Pj > Tj then 

p'j ■=Pj- ^\rj-x := Tj 
else 

p'j ■.= pj-rj_i := Tj + 1 
end if 
end for 



Algorithm 2 Second form of the deletion algorithm 
for j :— n downto i + 1 do 

p'j ■= Pj 
end for 

r := Pi 

for j :— i — 1 downto 1 do 
if Pj > r then 
p'j ■.= pj - 1 
else 
p'j := Pj 
if r < k then 

r :— r + 1 
end if 
end if 
end for 
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The transducer begins in a 'picking' state 0. Once it picks a letter to delete 
it passes through a sequence of states numbered according to the variable r 
in Algorithm 2. The transducer for the case A; = 3 is shown in Figure 2.1. 



Note 1 Strictly speaking, what we have constructed is a transducer for a 
relation where the words in question are read from right to left. To avoid 
notational clutter we make the convention that all finite automata and trans- 
ducers read their input from right to left. Of course any conclusion that we 
reach of the form "£ is a regular language" is independent of the direction 
of reading since L is regular if and only if its reverse is regular. 
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3,2 



Figure 1: Deletion transducer with k = ?> 



Proposition 4 Let C C E{Qk) be regular. Then each of the following sub- 
sets is also regular, and finite automata recognising them are effectively com- 
putable from an automaton recognising C 

1. {dip \ p & C,l < i < \p\}, 

2. {p e -E'(Qfc) I diP e jC, for some i}, 

3. {p e E{Vtk) I diP e L, for all i}. 



11 



Proof: The first set is CD and the second is CD* both of which are regular 
by Proposition 2. The third set is 

{p I diP ^ C for some i}*^ = {p \ diP G C'^ for some i}'^ 

Since regularity is preserved by complements the result follows again from 
Proposition 2. ■ 



2.2 A transducer for deleting any number of letters 

Again let tt = 7ri7r2 . . . 7r„ be a permutation in Vlk and let p = pip2 . . . Pn be its 
rank encoded form. We shall generalise the process described in the previous 

subsection so that it now deletes any number of letters (choosing which ones 
to delete non-deterministically again). From the resulting algorithm we shall 
be able to infer the existence of a transducer that defines the relation 

Ti. — {{p,p') I p' arises by deleting any number of letters from p} 

In the generalisation a right to left scan takes place as before. But now, 
rather than setting up a single variable r when the deleted letter is met, we 
have to set up a different variable every time we come to a letter that is to 
be deleted. 

So, suppose we come to a letter pa that we intend to delete. Then we define 
a variable r{d) (whose initial value will be pa) which will play the same role 
as the variable r in the previous section. Just as before when we process a 
letter pj (either to delete it or compute the value of pj) we shall have r{d) 
equal to the rank of tt^ in the set {tTj+i, . . . , 7r„} (that is, r{d) is the number 
of symbols in this set that are less than or equal to tt^). 

Processing a particular pj is then done as follows: 

1. if Pj is to be deleted we set up a variable r{j) as just mentioned and 
update any existing variables r{d); this updating is explained below. 

2. if Pj is not to be deleted we must use the variables r{d) so far defined 
to compute the value of p'j] and we must update these variables as 
necessary (see below). 



12 



Exactly as before, because of the meaning of each r{d) we have ttj > tt^ if 
and only if pj > r{d). Therefore the number of d's for which this occurs is 
the decrement that has to be applied to pj to obtain p'y 

To do the updating of the variable r{d) (so that it has the appropriate value 
when j is decreased by 1) we notice that any d for which pj > r{d) means 
that r((i) is not changed; otherwise it must be increased by 1. 

The behaviour of this algorithm when a symbol pj is processed is governed 
by the values of the set of variables r{d). In order to turn the algorithm into 
a transducer to recognise the relation Ti we have to demonstrate that only a 
fixed number of variables taking a fixed set of values is required. 

First, we have the same remark as before: any r{d) which reaches the value 
k can never affect whether pj should be changed; so such r{d)^s can be dis- 
carded. The second remark is that the r{d) are ranks of different elements 
within the same set ({tTj+i, . . . , 7r„}); therefore the values r{d) are distinct 
and so we shall never have more than A; — 1 of them to store. 

The state of the algorithm, as represented by the values of the r{d), is there- 
fore confined to one of a finite number of possibilities. A convenient way of 
representing the state (0,1) vector (si, . . . , Sfc„i). We set = 1 if 

there is a variable r{d) in the current 'live' set whose value is t; otherwise we 
set St = 0. This coding of state allows the automatic 'dropping' of a variable 
r{d) once it reaches the value k. 

Translating the way in which the r{d) are handled, the updating of the vari- 
ables Sf when a symbol pj = e is processed is easily seen to be: 

(Sl, . . . , Sk-l) := (Si, . . . , Se-l, 1, Sg) • • • ) 

if Pj is to be deleted and 

(Si, . . . , Sfc-l) '■= (■Sl) • • • ) ■Se-l5 0, Sg) • • • ) Sk-2) 

otherwise. The value of p'j in the latter case is pj — J2f<e^f- 
We summarise this discussion in 

Proposition 5 There is a transducer that defines the relation 

H. = {{p,p') I p' arises by deleting any number of letters from p} 
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Figure 2: Involvement transducer with k = 3 



The state diagram for the transducer in the case k — 3 is shown in Figure 2. 

Clearly is the relation of involvement on coded permutations and to reflect 
this we write p' < p if p' can obtained from p by deleting any number of 
letters. 

2.3 Regularity results 

In this subsection we state and prove the main results on A;-bounded classes. 



Theorem 1 There is an algorithm which decides whether or not a given 
regular set C C [k]* can be expressed as C — E{X) for some closed set of 
permutations A" C Jlj.. 



Proof: First note that a set X of permutations is closed if and only if for 
every tt = 111112 . . . 7r„ G A" and every i = 1, . . . , n, we have tt \ tTj G X. Thus, 
C = E{X) for some X if and only if {diP \ p e C,l < i < \p\} C C C E{nk)- 
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All the three above sets are regular (Proposition 3 and Proposition 4). and 
the automata accepting them are known, and hence we can decide whether 
these inclusions hold. ■ 



Theorem 2 A closed subset offlk is regular if and only if its basis is regular. 

Proof: Let X he a, closed set with basis B. Suppose first that X is regular. 
By definition B is the set of all permutations tt = tti . . . 7r„ such that n ^ X 
but TT \ TTj e A" for alH = 1, . . . , n. Thus 

E{B) = (E(X)f n {p I diP e E(X), for all i}, 

which is a regular set by Proposition 4. 

For the converse assume that B is regular. By Proposition 2 the set 
E(B)n^ ^{p\p' <p for some p' e E(B)} 

is regular and so its complement 

{E{B)n^f ^{p\p' for all p' e E{B)} 

is also regular. Therefore {E{B)H*)'^ fl E{Qk) is regular as well; but this set 
is E{X) itself. ■ 

The regular set operations that we have used (intersection and complemen- 
tation) are effectively computable in the sense that automata to recognise 
the resulting languages can be constructed. Therefore we have 

Corollciry 1 There is an algorithm which, given an automaton accepting 
E{X) for some regular closed set X , computes an automaton accepting E{B), 
where B is the basis of X. The converse is also true. 

This, in turn has the following pleasing consequence: 

Corollary 2 It is decidable whether or not a given regular closed set is 
finitely based. 

Corollary 3 The following are true for any closed set X C. fl). with a regular 
( in particular, finite ) basis: 
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(i) the enumeration sequence for X satisfies a linear recurrence with con- 
stant coefficients; 

(ii) membership in X can he checked in linear time. 

Proof: (i) X is in one-to-one length preserving correspondence with E{X^ 
which, being regular, has a rational generating function. 

(ii) Both testing for membership in a regular language and the process of 
encoding permutations are linear. ■ 

The first part of this corollary provides a partial (affirmative) answer to 
a conjecture of Gessel (that all finitely based closed sets have holonomic 
generating functions). 

Theorem 2 allows us to give exphcit examples of non-regular closed sets. Let 
A be any infinite antichain of permutations contained in Qj^. An example of 
such an antichain with A; = 3 is given in [2]. Let — {p'-n-i_i Oini-, • • •} be an 
infinite subset of A such that 

1. IttnJ = Hi, 

2. rii < n2 < . . . is not a finite union of arithmetic progressions. 

Then is a non-regular infinite antichain and, by Theorem 2, defines a 
closed set that is not regular. 

3 Monotone segment sets 

In this section we consider permuting machines with an unbounded memory. 
The memory is represented by a two-way infinite tape on which is stored 
an input sequence 1,2, ... ,n, one token per tape square, and a reading head 
moves up and down the tape. Wc consider machines M^j, which operate under 
a fixed regime of forward and backward scans of the tape that is specified by 
a sequence 4> — fif2 ■ ■ ■ fk + and — signs. 

The machine carries out k scans of the tape at the end of which all the 
input symbols will have been output. The ith scan is from left to right if 
fi = + and from right to left if /j = — . During each scan the machine will 
either skip over a symbol or output it (sequentially onto a second tape say) . 
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Such a computation can be described by a computation word Ci . . . c„ with 
1 < Ci < k; the term q gives the scan number on which symbol i was output. 

Example 3 Let = (+,—,—) so that does one left to right scan and 
two scans right to left. Suppose that the input tape contains 123456789. 
Then, supposing is subject to no further constraints it might, in its 
first scan output 2,4,8, in its second scan output 7,3, and in its final scan 
output 9,6,5,1. The result is the output permutation 248739651. Notice 
that there is another computation by this machine that produces the same 
output permutation (the first scan outputs 2,4, the second scan outputs 
8,7,3, and the third scan outputs 9,6,5,1). The computation words for 
these two computations are 312133213 and 312133223. 

Clearly this machine can only output permutations which have a segmenta- 
tion a/?7 where a is increasing and 7 arc decreasing. However, we do not 
exclude the possibility that, due to further constraints on the operation of 
the machine, not all permutations of this form can occur. 

In the general case the (closed) set of permutations output by is a subset 
of 

= {cricr2 • • • CTfc} 

where each cTj is an increasing sequence of symbols if /j = + and a decreasing 
sequence otherwise. The main results of this section are that the closed 
subsets of Wcj) have linear time recognisers and rational generating functions. 

Every computation word c gives rise to a permutation D^{c) e W(j,. To be 
precise, if we regard c as a function 

c : [n] ^ [k\ 

then D^{c) is the permutation obtained by concatenating the sets £"^(1) 
through c^^(fc), with the ith set in this concatenation arranged in increasing 
order if /j = +, and in decreasing order ii fi — —. It is easily seen that 
is the image of [A;]* under the map D^. 

We have observed already that D,^ is not one-to-one but clearly each D^^(7r) 
is a finite set (that is, every permutation tt e can be obtained in only 
finitely many ways). We shall find it convenient to call its members the 
encodings of tt. This situation differs from that in the previous section in 
that now a permutation may have several encodings. Nevertheless we define 
subset X of W(f, to be regular, if D'7^{X) is a regular subset of [k]*. 
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Lemma 2 Suppose that s,p & [k]* and s is a suhword of p. Then D^{s) ^ 
D^{p). Also suppose that a ^ tt are elements ofWcf,. Then for each encoding 
p of t: there exists an encoding s of a which is a suhword of p 

Proof: The first part is immediate. For the remainder, take a subset of the 
positions in tt with pattern a. Then just take s to be the subword of p on 
the same positions. ■ 

Theorem 3 Every closed subset of is regular. 

Proof: Let X he a, closed subset of and let B be its basis. By Theorem 
2.9 of [2] B is finite. Let B be the set of all elements of [A;]* which have a 
subword belonging to D'^^{B). Since D~^^{B) is finite, B is regular. Suppose 
that TT e A". Then no encoding p of tt can contain an element s of D'^^{B) 
as a subword, for otherwise D^{s) ^ tt. So D^^{X) C B'^. On the other 
hand, if p G B'^, and tt = D^{p), then tt G A' - for if not there is some 
a E B with a ^ n, and then some encoding s of cr which is a subword of p, 
a contradiction. So D'^^{X) = B'', which is regular. ■ 

Corollary 4 There is a linear time recognition algorithm for any closed sub- 
set ofW^. 

We cannot immediately deduce that every closed subset of Wtj, has a rational 
generating function since the correspondence between W,/, and [k\* is not 
one-to-one. To get around this difficulty we define, for every a G W^, a 
distinguished encoding as follows. Let ai be the longest monotone initial 
segment of a consistent with the sign /i. Having chosen ci we choose the 
next monotone segment (T2 (corresponding to also as long as possible, and 
we continue in this manner until all of a has been segmented (necessarily with 
k or fewer segments). The corresponding encoding Ci . . . c^, where q = j if 
i G CTj, is called the greedy encoding of a. (In the example above the first 
encoding was greedy, the second was not). 

Lemma 3 The greedy encoding of is a regular set. 

Proof: Let p and q = p-\-lhe any two consecutive positions of 0. In the 
greedy encoding ci . . . c„ of a permutation a G W<^ let the positions where 
Ch — p he h = ii,i2, ■ ■ . ,ia and those where Ch — q he h — ji, J2 • • • , jb- The 
greedy condition implies one of the following: 
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[fp — +, fq — + ] Since a has adjacent segments ii, i2, ■ ■ ■ ,ia and ji, ■ ■ ■ ,jb 
we have ia > ji, that is, in c, the final p comes after the first q. 

[fp ^ +,fq^ - ] Here a has adjacent segments ii, i2,...,ia and jb, j2, ji, 
so ia > jb', that is, the final p comes after the final q. 

[fp = —, fq = + ] Similarly, the first p comes before the first q. 

[fp=—,fq = — ] The first p comes before the last q. 

Every consecutive p,p + 1 gives a restriction on the form of a greedy en- 
coding but these restrictions are all recognisable by a finite automaton thus 
completing the proof. ■ 

Theorem 4 Every closed subset of W<^ has a rational generating function. 

Proof: Let X be any closed subset of W^. By Theorem 3 D^^{X) is regular 
and therefore D'^^{X) fl where Q is the set of greedy encodings of W^, is 
also regular. But this set is in one-to-one correspondence with X. u 



4 Final remarks 

We have shown that closed sets are the natural objects to study in the analy- 
sis of permuting machines. We have also demonstrated that, when a suitable 
encoding of permutations is available, finite automata are a powerful tool in 
this study. Nevertheless many problems remain. In particular, one natural 
question is how one might extend the automata tools to use context-free 
encodings. Here one might hope to prove that certain closed sets have an 
algebraic generating function rather than a rational one. A natural candi- 
date to investigate are the closed subsets of A{312) (which is the language 
associated with a single stack) for here well-formed bracket sequences encode 
permutations in a natural way. We hope to report progress on such problems 
in a subsequent paper. 

Another issue is that of practicabihty. The "effective" methods we have 
developed for contructing automata frequently lead to automata with very 
large numbers of states since, in particular, we often need to convert a non- 
deterministic automaton to its deterministic version. In some special cases 
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we have managed to contain this state explosion and have carried out these 
constructions, and this gives hope that more efficient methods may exist. 
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